Search Results: "Michael Stapelberg"

8 June 2013

Michael Stapelberg: Uploading packages via SFTP

Yesterday I uploaded a big package and got multiple timeouts. I then figured out that DDs can also upload using SFTP (i.e. SSH s file transfer thingie) instead of traditional FTP, which seems like a more modern alternative. So let s give that a try. With dput-ng, the following configuration leads to using sftp by default:
mkdir -p ~/.dput.d/profiles/
cat > ~/.dput.d/profiles/ftp-master.json <<EOT
 
    "fqdn": "ssh.upload.debian.org",
    "incoming": "/srv/upload.debian.org/UploadQueue/",
    "method": "sftp"
 
EOT
Note that uploading via SFTP will lead to debianqueued uploading the files via FTP for you. But maybe that is more reliable than doing it yourself. We ll see :-).

27 May 2013

Michael Stapelberg: Results of the Debian systemd survey

A week ago, we started the Debian systemd survey. The goal was to figure out a few trends and answer the following two questions:
  1. Do our subjective impressions from the discussions on debian-devel reflect the general sentiment about systemd?
  2. What are the main concerns that most people have?
Thank you all for your participation! General I am happy to tell that 2113 people had a look at the survey and 573 people actually participated. Of those, 45.7% said they are a DD, DM or otherwise maintaining packages. 74.5% said they actually booted and used a computer running systemd. With regards to having systemd in Debian at all, not necessarily as the default init system, answers are as follows: 43.9% said they personally want systemd as the default init system in Debian, while 32.2% don t want that. The remaining 23.7% don t know yet. Top concerns About 50% of participants provided one or more concerns. Evaluating this part of the survey is obviously the hardest, since the answers were provided as free text. I tried categorizing the feedback into a few buckets. Each bucket comes with a number which is a weighted (!) count, i.e. the top concern counts 3 times, the second concern counts 2 times and the last concern just once. If you cannot find your personal concerns, that is because those which were just voiced by less then five people are not listed here. As I said, we cared about trends in this survey, not individual opinions.
  1. systemd is too complex, or bloated, or it does too many things, or has too many dependencies (weight: 217)
  2. systemd is not portable to non-linux systems, e.g. Debian/kFreeBSD or HURD (weight: 199)
  3. Debugging the boot process is harder than in sysvinit (weight: 106)
  4. I have a problem with systemd upstream and/or Lennart in particular (weight: 87)
  5. systemd violates the UNIX philosophy (weight: 35)
  6. I dislike binary logs and/or the journal in general (weight: 30)
  7. systemd is too new, still untested, unstable and experimental (weight: 24)
  8. I cannot find enough documentation in general or about the transition from sysvinit to systemd in particular (weight: 20)
  9. People need to learn new commands/how the new system works (weight: 20)
  10. I don t know what problems systemd solves and/or have no problems with sysvinit currently (weight: 19)
  11. I think the configuration is binary and/or the configuration format is weird (weight: 13)
  12. systemd s code is not good (memory leaks, performance problems) and/or upstream s development lacks best practices such as long-term stable branches (weight: 9)
  13. There are problems with systemd in Debian (e.g. Debian-specific extensions to /etc/fstab not supported, system doesn t boot) (weight: 9)
  14. Customizing the boot is harder than with sysvinit (weight: 8)
  15. systemd is not compatible with sysvinit (weight: 8)
I realize that the choice of buckets is somewhat broad and there are many different, finely nuanced opinions out there. Again, this is for seeing a trend, not accounting for every single individual opinion. In case you strongly believe that I did something wrong when evaluating the survey results, please contact me and we can work something out. Conclusions/Actions I know this is a controversial topic. Please don t start yet another systemd discussion on debian-devel. We, the systemd Debian maintainers, will try to come up with good answers to all listed concerns and communicate them in a friendly and concise way soon. Furthermore, we have recognized the need for more documentation/information about how things are supposed to work with regards to the transition (and systemd in general) and will address that, too. Raw questions/numbers
Total records in survey: 573
Are you a Debian Developer (DD) or Debian Maintainer (DM) or otherwise currently maintaining Debian packages?
262 Yes
311 No
Did you ever boot and use a computer running systemd? It does not matter whether that computer was running Debian or a different operating system.
427 Yes
146 No
What is your general sentiment towards having systemd in Debian (not necessarily as default)?
358 I welcome systemd in Debian, everything is fine.
 81 I am not sure yet.
 46 I don t care.
 88 I don t want systemd in Debian.
Would you personally want systemd as the default init system in Debian?
252 Yes
136 I don t know
185 No

25 May 2013

Michael Stapelberg: Cloning git-buildpackage repositories

Whenever I want to work on some package, I usually clone its git repository, make my changes, then push and upload the Debian package. I don t keep those repositories around in order to avoid cruft and also to have a 100% clean, up-to-date setup whenever I start working on something. Everytime I clone such a repository, I struggle with the setup. For example, I usually forget the --pristine-tar flag for gbp-clone. Also, I usually forget to push other branches (working on debian , forgetting upstream ) and, even more often, I forget pushing tags. I spent some time on this and figured out that one can use the following to make --pristine-tar the default:
cat >> ~/.gbp.conf <EOF
[DEFAULT]
pristine-tar = True
EOF
Avoiding my other pain points is not so easy apparently, so I wrote a little shell function (only tested with zsh!) which uses debcheckout to get the git URL, gbp-clone to actually clone it and a few git config calls to make me able to just git push when I am done and not worry about anything:
# Clones the git sources of a Debian package
# needs debcheckout from devscripts and gbp-clone from git-buildpackage
function d-clone()  
    local package=$1
    if debcheckout --print $package >/dev/null
    then
        set -- $(debcheckout --print $package)
        if [ "$1" != "git" ]
        then
            echo "$package does not use git, but $1 instead."
            return
        fi
        echo "cloning $2"
        gbp-clone $2   return
        # Change to the newest git repository
        cd $(dirname $(ls -1td */.git   head -1))   return
        # This tells git to push all branches at once,
        # i.e. if you changed upstream and debian (after git-import-orig),
        # both upstream and debian will be pushed when running  git push .
        git config push.default matching   return
        # This tells git to push tags automatically,
        # so you don t have to use  git push; git push --tags .
        git config --add remote.origin.push "+refs/heads/*:refs/remotes/origin/*"   return
        git config --add remote.origin.push "+refs/tags/*:refs/tags/*"   return
        echo "d-clone set up everything successfully."
    else
        echo "debcheckout $package failed. Is $package missing Vcs tags?"
    fi
 
With that function, starting work on a package becomes as easy as d-clone golang-doc .

20 May 2013

Michael Stapelberg: Debian systemd survey

In the past, we have had multiple heated discussions involving systemd. We (the pkg-systemd-maintainers team) would like to better understand why some people dislike systemd. Therefore, we have created a survey, which you can find at http://survey.zekjur.net/index.php/391182 Please only submit your feedback to the survey and not this thread, we are not particularly interested in yet another systemd discussion at this point. The deadline for participating in that survey is 7 days from now, that is 2013-05-26 23:59:00 UTC. Please participate only if you consider yourself an active member of the Debian community (for example participating in the debian-devel mailing list, maintaining packages, etc.). Of course, we will publish the results after the survey ends. Thanks! Best regards,
the Debian systemd maintainers

12 May 2013

Ian Campbell: qcontrol 0.5.1

I've just released qcontrol 0.5.1. Changes since the last release: I also put together a very basic homepage. Get it from gitorious or http://www.hellion.org.uk/qcontrol/releases/0.5.1/. The Debian package will be uploaded shortly.

20 April 2013

Ulrich Dangel: Analyzing rc bug messages

Michael Stapelberg recently posted a blog post about looking into the number of Debian Developers actively working on RC bugs for the upcoming wheezy release. In this blog post I analyze the data shared by Michael and provide the R commands used to generate the plots & findings. If you are interested into looking into the data yourself, but don t like R, I suggest using ipython notebook + numpy instead.

Analysis After parsing the data file we typically want to get an understanding of the data, by using summary(bugs) we get the minimum(1), median(5), mean(15.4), max(716) and quantiles of the data. This shows that the number of messages is wide-spread and a few people contribute a lot. To visualize the dispersion of the data we can create a box plot showing the range of messages: boxplot As the first and third quantile are close together we can assume that the majority of the work is done by a few, especially since the second quantile is 5. This is supported by the histogram below, where the x axis is the number of recorded messages and y is the number of developers. histogram

Top 10 contributors The TOP 10 contributors, according to the dataset, are:
  1. Lucas Nussbaum - 716 messages
  2. Gregor Herrmann - 270 messages
  3. Jakub Wilk - 270 messages
  4. Andreas Beckmann - 225 messages
  5. Julien Cristau - 205 messages
  6. Cyril Brulebois - 169 messages
  7. Moritz Muehlenhoff - 162 messages
  8. Michael Biebl - 159 messages
  9. Salvatore Bonaccorso - 158 messages
  10. Christoph Egger - 142 messages

r commands These are the commands used to generate the plots and information in this plot:
bugs <- read.csv("by-msg.csv")
summary(bugs)
boxplot(bugs$rcbugmsg, log='y', range=0, ylab="# bugs")
quantile(bugs$rcbugmsg)
0%  25%  50%  75% 100%
1    2    5   12  716
# create histogram
llibrary('ggplot2')
ggplot(bugs, aes(x=rcbugmsg)) + geom_histogram(binwidth=.5, colour="black", fill="black") + scale_x_sqrt()
top10 <- tail(bugs[order(bugs$rcbugmsg),], 10)
top10

Ulrich Dangel: Analyzing rc bug messages

Michael Stapelberg recently posted a blog post about looking into the number of Debian Developers actively working on RC bugs for the upcoming wheezy release. In this blog post I analyze the data shared by Michael and provide the R commands used to generate the plots & findings. If you are interested into looking into the data yourself, but don t like R, I suggest using ipython notebook + numpy instead.

Analysis After parsing the data file we typically want to get an understanding of the data, by using summary(bugs) we get the minimum(1), median(5), mean(15.4), max(716) and quantiles of the data. This shows that the number of messages is wide-spread and a few people contribute a lot. To visualize the dispersion of the data we can create a box plot showing the range of messages: boxplot As the first and third quantile are close together we can assume that the majority of the work is done by a few, especially since the second quantile is 5. This is supported by the histogram below, where the x axis is the number of recorded messages and y is the number of developers. histogram

Top 10 contributors The TOP 10 contributors, according to the dataset, are:
  1. Lucas Nussbaum - 716 messages
  2. Gregor Herrmann - 270 messages
  3. Jakub Wilk - 270 messages
  4. Andreas Beckmann - 225 messages
  5. Julien Cristau - 205 messages
  6. Cyril Brulebois - 169 messages
  7. Moritz Muehlenhoff - 162 messages
  8. Michael Biebl - 159 messages
  9. Salvatore Bonaccorso - 158 messages
  10. Christoph Egger - 142 messages

r commands These are the commands used to generate the plots and information in this plot:
bugs <- read.csv("by-msg.csv")
summary(bugs)
boxplot(bugs$rcbugmsg, log='y', range=0, ylab="# bugs")
quantile(bugs$rcbugmsg)
0%  25%  50%  75% 100%
1    2    5   12  716
# create histogram
llibrary('ggplot2')
ggplot(bugs, aes(x=rcbugmsg)) + geom_histogram(binwidth=.5, colour="black", fill="black") + scale_x_sqrt()
top10 <- tail(bugs[order(bugs$rcbugmsg),], 10)
top10

30 March 2013

Michael Stapelberg: Analyzing RC bug messages

Recently, I was wondering how many Debian Developers are actively working on RC bugs in some way or another in the time period of the last release (squeeze) to now (shortly? before wheezy). I therefore grabbed the mailing list archives of debian-bugs-dist@ from gmane, used only those messages whose X-Debian-PR-Message header matches an RC bug (list retrieved from UDD) and then attributed the message counts to the appropriate Debian Developer. I am sure that there are subtle mistakes in the data I retrieved and that there most likely is a better way to achieve the same results, but this is only to get a trend and should be good enough for that. So, it turns out that 514 different Debian Developers have sent messages regarding RC bugs since squeeze. That s about half of our 985 total active Debian Developers. I would love to show a good histogram of the actual message counts (not sure how well received showing the raw data is ), but I suck at visualizing such data in a compact way. Is there anyone familiar with R (or other free data visualization tools) and willing to help? I can send you the CSV file, just send me an email to stapelberg@.

8 March 2013

Michael Stapelberg: Replying to Debian BTS messages in notmuch

Previously, my workflow regarding replying to bugreports outside my own packages was very uncomfortable: I first downloaded the mbox archive from the BTS, then imported that in claws-mail, hit reply all, remove submit@, add bugnumber@, then send the email. Therefore, I decided to hack up a little elisp function to automate this process for notmuch. It first downloads the message from the BTS, adds it to the notmuch database, then calls notmuch-mua-reply on the message and fixes the To: header:
;; Removes submit@bugs.debian.org from the recipients of a reply-all message.
(defun debian-remove-submit (recipients)
  (delq nil
	(mapcar (lambda (recipient)
		  (and (not (string-equal (nth 1 recipient) "submit@bugs.debian.org"))
		       recipient))
		recipients)))
(defun debian-add-bugrecipient (recipients bugnumber)
  (let* ((bugaddress (concat bugnumber "@bugs.debian.org"))
	 (addresses (mapcar (lambda (x) (nth 1 x)) recipients))
	 (exists (member bugaddress addresses)))
    (if exists
	recipients
      (append (list (list (concat "Bug " bugnumber) bugaddress)) recipients))))
;; TODO: msg should be made optional and it should default to the latest message in the bugreport.
;; NB: bugnumber and msg are both strings.
(defun debian-bts-reply (bugnumber msg)
  ;; Download the message to ~/mail-copy-fs/imported.
  (let ((msgpath (format "~/mail-copy-fs/imported/bts_%s_msg_%s.msg" bugnumber msg)))
    (let* ((url (format "http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=%s;mbox=yes;bug=%s" msg bugnumber))
	   (download-buffer (url-retrieve-synchronously url)))
      (save-excursion
	(set-buffer download-buffer)
	(goto-char (point-min)) ; just to be safe
	(if (not (string-equal
		  (buffer-substring (point) (line-end-position))
		  "HTTP/1.1 200 OK"))
	    (error "Could not download the message from the Debian BTS"))
	;; Delete the HTTP headers and the first "From" line (in order to
	;; make this a message, not an mbox).
	(re-search-forward "^$" nil 'move)
	(forward-char)
	(forward-line 1)
	(delete-region (point-min) (point))
	;; Store the message on disk.
	(write-file msgpath)
	(kill-buffer)))
    ;; Import the mail into the notmuch database.
    (let ((msgid (with-temp-buffer
		   (call-process "~/.local/bin/notmuch-import.py" nil t nil (expand-file-name msgpath))
		   (buffer-string))))
      (notmuch-mua-reply (concat "id:" msgid) "Michael Stapelberg <stapelberg@debian.org>" t)
      ;; Remove submit@bugs.debian.org, add <bugnumber>@bugs.debian.org.
      (let* ((to (message-fetch-field "To"))
	     (recipients (mail-extract-address-components to t))
	     (recipients (debian-remove-submit recipients))
	     (recipients (debian-add-bugrecipient recipients bugnumber))
	     (recipients-str (mapconcat (lambda (x) (concat (nth 0 x) " <" (nth 1 x) ">")) recipients ", ")))
	(save-excursion
	  (message-goto-to)
	  (message-delete-line)
	  (insert "To: " recipients-str "\n")))
      ;; Our modifications don t count as modifications.
      (set-buffer-modified-p nil))))
In case you want to get updates, you can find the latest version of this code in my configfiles git repository. To add a single message to the notmuch database and get its message ID, I have written this simple python script (using python-notmuch), located in ~/.local/bin/python-import.py:
#!/usr/bin/env python
# vim:ts=4:sw=4:et
import notmuch
import sys
if len(sys.argv) < 2:
    print "Syntax: notmuch-import.py <filename>"
    sys.exit(0)
db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
(msg, status) = db.add_message(sys.argv[1])
print msg.get_message_id()
If you have any improvements, I d love to hear about it. If it s useful for you, enjoy.

6 February 2013

Michael Stapelberg: RC bugs

I recently worked on the following RC bugs: By the way, in case anyone needs to reproduce an armhf bug and wants to do so in a chroot with qemu, here are the steps I used:
sudo qemu-debootstrap --arch=armhf --foreign wheezy armhf http://ftp.de.debian.org/debian
sudo LC_ALL=C chroot armhf /bin/bash
echo 'deb http://ftp.de.debian.org/debian wheezy main' > /etc/apt/sources.list
echo 'nameserver 8.8.8.8' > /etc/resolv.conf
apt-get update
apt-get install clang
Also, here are UDD queries which I prefer to those posted by RichiH. Note that they don t display all bugs, but ignore those which were created in the last 7 days.

5 February 2013

Michael Stapelberg: RC bugs

I recently worked on the following RC bugs: It is getting hard to find RC bugs to work on. If you have access to ia64 hardware, have a look at #694971 Epiphany browser crashes within JSC::JSArray::increaseVectorLength() or #697172 sporadic crashes of epiphany browser due to a thread-unsafe favicon database. Both have patches available that just need to be tested.

2 January 2013

Michael Stapelberg: WIT in Debian

I just uploaded wit-2.10a to Debian experimental (it has to pass the NEW queue first, though). WIT (Wiimms ISO Tools) is a set of command-line tools to manipulate Wii and GameCube ISO images and WBFS containers. It is useful (for me) to store backups of my Wii games on a USB hard disk drive. This saves me the optical disc juggling, doesn t wear off the discs as fast and gives faster load times. Here is an example session where I format one partition of the USB hard disk with WBFS and then copy my old WBFS image over to it:
$ wwt format -v --force /dev/sde3
wwt: Wiimms WBFS Tool v2.10a r0 x86_64 - Dirk Clemens - 2013-01-02
FORMAT BLOCK DEVICE /dev/sde3 [172 GiB, hss=512]
** 1 file formatted.
$ wwt add --part /dev/sde3 /media/sde1/wbfs/The\ Legend\ of\ Zelda\ Skyward\ Sword\ \[SOUP01\]/*.wbfs
*****  wwt: Wiimms WBFS Tool v2.10a r0 x86_64 - Dirk Clemens - 2013-01-02  *****
WBFSv1 #1/1 opened: /dev/sde3
 - ADD 1/1 [SOUP01] WBFS:/media/sde1/wbfs/The Legend of Zelda Skyward Sword [SOUP01]/SOUP01.wbfs/#0
* WBFS #1: 1 disc added.
wwt add --part /dev/sde3   0,02s user 7,66s system 2% cpu 5:02,68 total
$ wwt list
ID6     1/500 discs (4 GiB)
---------------------------------------------------------------------
SOUP01  The Legend of Zelda Skyward Sword
---------------------------------------------------------------------
Total: 1/500 discs, 4176 MiB ~ 4 GiB used, 171444 MiB ~ 167 GiB free.

5 December 2012

Michael Stapelberg: RC bugs

I recently worked on the following RC bugs:

4 December 2012

Michael Stapelberg: RC bugs

I recently worked on the following RC bugs:

3 December 2012

Michael Stapelberg: RC bugs

I recently worked on the following RC bugs:

Michael Stapelberg: RC bugs

I recently worked on the following RC bugs:

1 December 2012

Michael Stapelberg: rc bug #694670: kscd does not play any CD

Dear Lazyweb, I recently investigated #694670 - kscd does not play any CD essentially, at least on my machine, phonon-based applications such as kscd won t play Audio CDs when your CD drive is not called /dev/cdrom, but e.g. /dev/cdrom5 because I have had multiple CD drives connected to my machine. Now, I am not very deeply involved into KDE or phonon in particular. Therefore, I am asking: Is there anyone who could have a look into this issue? It should be easy to reproduce, you can simply append "5" to "cdrom" in /etc/udev/rules.d/70-persistent-cd.rules and reboot afterwards. As an outsider, I am somewhat baffled by the complexity of C++, phonon, gstreamer, etc. All these layers of abstraction are not that easy to debug, at least not for me. Good blog posts about digging into this issue are appreciated :-).

25 November 2012

Michael Stapelberg: RC bugs

I recently worked on the following RC bugs: I have to say that I find the BTS to be very hard to read from the perspective of a newcomer who wants to help squashing RC bugs, both from a technical and a social standpoint. As an example for the technical side: blocking bugs are very easy to miss, so innocent bugreports which seem actionable turn out to just wait for another bug with lots of discussion. Also, it seems like the best interface to the BTS for RC bugs is UDD, so RichiH s RC report was quite helpful. For the social side: Bugreports often are in a state in which it is not clear what the next course of action is, e.g. #634261 (the bug is not considered RC, but still has severity: grave) or #684633 (dropping from testing suggested but not clear whether a removal request has been filed or not). I m aware that this is very vague critic. I suppose what I am missing is a way in which we can make the list of RC bugs more actionable. When I am browsing through RC bugs, I often open bugreports and figure out that there s nothing I can do. Possibly a way to improve this situation is by letting people claim ownership of a bug and then filtering for un-owned bugs and those which have an owner but have not seen any activity for some period of time.

7 November 2012

Pablo Lorenzzoni: That s a lot to do!

Reading about Michael Stapelberg s codesearch I bet a lot of people had the same idea. I just had to post a screenshot of it: It seems we have a lot to fix :-)

6 November 2012

Michael Stapelberg: Debian Code Search

I hereby announce a new Debian project: Debian Code Search. Debian Code Search is a search engine for program source code within Debian.
It allows you to search all 17000 source packages, containing 130 GiB of
FLOSS source code (including Debian packaging) with regular expressions. You can use the search engine at http://codesearch.debian.net/
Here are a few sample queries: The corresponding thesis (and source code, of course) will be released soon
(2013-01-15 being the deadline, but I hope I can do it earlier).

Next.

Previous.